This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space. This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics. Our main result, the Lower Confidence-based Continuous Control (LC3) algorithm, enjoys a near-optimal $O(\sqrt{T})$ regret bound against the optimal controller in episodic settings, where $T$ is the number of episodes. The bound has no explicit dependence on dimension of the system dynamics, which could be infinite, but instead only depends on information theoretic quantities. We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.

Additional Feedback: 1. Could you please define'continuous control' in the first sentence of the Intro? Does this mean continuous-time continuous-space control? 2. Line 20: Please could you define this term'provably correct' as well? In control of uncertain dynamics, we rarely care about'correctness' and far more about'robustness' since even an arbitrarily small amount away from the exact nonlinear system could (in general) induce very different dynamics, so unless one can get an exact model, one typically relies on robust controllers for safety during implementation. There is a lot of work in robust control that could be (and has been) seamlessly integrated in very unknown, very safety-critical, and very complex environments (airplane/ship navigation/biomedicine) which work great without any RL. The assumptions and theorems, are, of course, quite different.

The introduced algorithm LC3 enjoys an O(sqrt{T}) regret bound against the optimal controller with no explicit dependence on the dimension of the system dynamics. The paper received a mostly positive evaluation from the reviewers with one vote below the acceptance threshold (scores of 7, 7, and 5). The main strengths of the paper were identified as: - Novel results (on of the first in adaptive non-linear control) which should be of interest to the NeurIPS community. Several weaknesses were also pointed out: - One of the reviewers found the contribution of the theoretical results to be marginal comparing to the past work.

This work studies the problem of sequential control in an unknown, nonlinear dynamical system, where we model the underlying system dynamics as an unknown function in a known Reproducing Kernel Hilbert Space. This framework yields a general setting that permits discrete and continuous control inputs as well as non-smooth, non-differentiable dynamics. Our main result, the Lower Confidence-based Continuous Control (LC3) algorithm, enjoys a near-optimal O(\sqrt{T}) regret bound against the optimal controller in episodic settings, where T is the number of episodes. The bound has no explicit dependence on dimension of the system dynamics, which could be infinite, but instead only depends on information theoretic quantities. We empirically show its application to a number of nonlinear control tasks and demonstrate the benefit of exploration for learning model dynamics.